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DETAILED ACTION 
Claim Objections 

1 . Claim 1 9 and 39 are objected to because of the following informalities: 

• Regarding claims 19 and 39, the phrase "depending one whether", should be - 
depending on whether-. 
Appropriate correction is required. 



DETAILED ACTION 



Claim Rejections - 35 USC § 102 



2. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1 ) an application for patent, published under section 1 22(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 



3 Claims 1, 4, 6-7, 11, 21, 24, 26-27, 31 and 41 are rejected under 35 

U.S.C. 102(e) as being anticipated by Cooley et al. (PGPUB 2005/0149336), hereinafter 

referenced as Cooley. 
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Regarding claims 1, 21 and 41, Cooley discloses a system, method and 
computer-readable medium, hereinafter referenced as a system, for indexing electronic 
information, comprising: 

an authoring module that coordinates an authoring procedure for creating an 
index file that includes pattern word sets corresponding to data objects (voice input 
associated with an image; column 1, paragraph 0012) stored in a memory device (figure 
2, element 204), said pattern word sets being generated with a speech recognition 
engine that transforms (translates) spoken data descriptions (voice input) into text data 
descriptions (to text), said pattern word sets being associated with data object identifiers 
that identify said data objects (image; column 1, paragraphs 0012 and 0016-0017); and 

a retrieval module that manages a retrieval procedure in which said speech 
recognition engine converts a spoken data request into a text data request (translates 
voice input to text), said retrieval module comparing said text data request and said 
pattern word sets to identify a requested object identifier for locating a requested data 
object from among said data objects stored in said memory device (once voice data has 
been translated to text, the text can be presented with the image on a display; column 1 , 
paragraphs 0012 and 0016-0017). 

Regarding claims 4 and 24, Cooley discloses a system wherein said data 
objects include stored images created by an imaging device (image; column 1, 
paragraph 0016), said stored images being saved in said memory device of a host 
electronic device (figure 2, element 204). 



Application/Control Number: 10/812,560 Page 4 

Art Unit: 2626 

Regarding claims 6 and 26, Cooley discloses a system wherein said authoring 
module displays thumbnail representations of said stored images on a display of said 
host electronic device to facilitate said authoring procedure (thumbnail; column 1, 
paragraph 15, columns 1-2, paragraph 0017 and column 4, paragraph 0033). 

Regarding claims 7 and 27, Cooley discloses a system wherein a selected 
image is chosen from thumbnail representations of said stored images for creating a 
corresponding one of said pattern word sets, said selected image being chosen by a 
system user after viewing said thumbnail representations on an indexing graphical user 
interface (thumbnail presented on display; column 1, paragraph 15, columns 1-2, 
paragraph 0017 and column 4, paragraph 0033). 

Regarding claims 11 and 31, Cooley discloses a system wherein said authoring 
module edits said text data descriptions on an indexing graphical user interface 
according to editing instructions from a system user to produce said pattern word sets, 
said editing instructions including a change-word instruction, a delete-word instruction, 
and an add-word instruction (modify, delete and add; column 4, paragraph 0036). 

4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 
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5 Claims 1-5, 8, 10, 12-14, 16, 20-25, 28, 30, 32-34, 36, 40-47 are rejected under 
35 U.S.C. 102(b) as being anticipated by Greene, Jr. et at. (USPN 6,377,925), 
hereinafter referenced as Greene. 

Regarding claims 1, 21 and 41, Greene discloses a system, method and 
computer-readable medium, hereinafter referenced as a system, for indexing electronic 
information, comprising: 

an authoring module that coordinates an authoring procedure for creating an 
index file that includes pattern word sets corresponding to data objects stored in a 
memory device, said pattern word sets being generated with a speech recognition 
engine (each word of text stored in database; column 5, lines 35-39) that transforms 
spoken data descriptions into text data descriptions (receive speech inputs from 
microphone and converts speech into text; column 4, line 65 - column 5, line 5), said 
pattern word sets being associated with data object identifiers that identify said data 
objects (column 6, lines 1-11); and 

a retrieval module that manages a retrieval procedure in which said speech 
recognition engine converts a spoken data request into a text data request (receive 
speech inputs from microphone and converts speech into text; column 4, line 65 - 
column 5, line 5), said retrieval module comparing said text data request and said 
pattern word sets to identify a requested object identifier for locating a requested data 
object from among said data objects stored in said memory device (column 6, lines 1-11 
with column 5, lines 24-30). 
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Regarding claims 2 and 22, Greene discloses a system wherein said pattern 
word sets each include one or more search terms derived from said text data 
descriptions, said search terms including one or more key words that relate to said data 
objects (search for word; column 7, lines 59-66 with column 8, lines 45-60). 

Regarding claims 3 and 23, Greene discloses a system wherein an indexing 
graphical user interface is utilized to create said pattern word sets, said indexing 
graphical user interface including an object section for displaying representations of said 
data objects (images are displayed on the monitor; column 5, lines 2-6), said indexing 
graphical user interface also including an editing section for converting said text data 
descriptions into said pattern word sets (column 5, lines 44-52). 

Regarding claims 4 and 24, Greene discloses a system wherein said data 
objects include stored images created by an imaging device (column 5, lines 2-6), said 
stored images being saved in said memory device of a host electronic device (figure 1, 
element 53). 

Regarding claims 5 and 25, Greene discloses a system wherein said authoring 
module instructs said host electronic device to enter an authoring mode for performing 
said authoring procedure in response to a verbal authoring-mode command that is 
recognized by said speech recognition engine (command; column 8, lines 2-30). 

Regarding claims 8 and 28, Greene discloses a system wherein said spoken 
data description is provided by a system user speaking into a sound sensor (figure 1, 
element 32 with column 7, lines 6-29 and column 8, lines 2-9) of said host electronic 
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device, said spoken data description including a verbal description of a selected image 
from said stored images ("the cow jumped over the moon"; column 5, lines 44-52). 

Regarding claims 10 and 30, Greene discloses a system wherein said authoring 
module displays said text data descriptions on an editing section of an indexing 
graphical user interface for editing into said pattern word sets (edited through keyboard; 
column 6, lines 18-20). 

Regarding claims 12 and 32, Greene discloses a system wherein said authoring 
module stores each of said pattern word sets along with a corresponding one of said 
data object identifiers into said index file for subsequently performing said retrieval 
procedure (column 6, lines 1-1 1 with column 5, lines 24-42). 

Regarding claims 13 and 33, Greene discloses a system wherein said retrieval 
module instructs said host electronic device to enter a retrieval mode for performing 
said retrieval procedure in response to a verbal retrieval-mode command that is 
recognized by said speech recognition engine (column 7, line 61 - column 8, line 45 
with column 9, lines 4-18). 

Regarding claims 14 and 34, Greene discloses a system wherein said spoken 
data request is provided by a system user speaking into a sound sensor of said host 
electronic device (column 7, lines 6-28 and column 8, lines 2-9), said spoken data 
request including a verbal request related to a selected image from said stored images 
(column 6, lines 1-11). 

Regarding claims 16 and 36, Greene discloses a system wherein said retrieval 
module accesses said index file for performing a search procedure that identifies said 
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requested data object by seeking search matches between said text data request and 
said pattern word sets (comparing text and accessing clip; column 5, lines 24-43). 

Regarding claims 20 and 40, Greene discloses a system wherein said retrieval 
module displays said requested data object on a display of said host electronic device 
after said retrieval procedure has been performed (retrieve word then display; column 8, 
lines 17-22 with column 7, lines 59-62). 

Regarding claim 42, it is interpreted and rejected for the same reasons as set 
forth in claims 1, 21 and 41. 

Regarding claim 43, Greene discloses a system for indexing electronic 
information, comprising: 

an authoring module that coordinates an authoring procedure for creating an 
index file that includes pattern word sets corresponding to images (column 5, lines 2-3) 
stored in a memory device (figure 1, element 53), said pattern word sets being 
generated with a speech recognition engine that transforms spoken data descriptions 
into text data descriptions (receive speech imputes from microphone and converts the 
speech into text; column 4, line 65 - column 5, line 5) that are edited upon an indexing 
graphical user interface (edit through keyboard; column 6, lines 18-20) for creating said 
pattern word sets, said pattern word sets being associated with image identifiers that 
identify said images (column 5, lines 2-3 with column 6, lines 1-11); and 

a retrieval module that manages a retrieval procedure in which said speech 
recognition engine converts a spoken data request into a text data request receive 
speech imputes from microphone and converts the speech into text; column 4, line 65 - 
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column 5, line 5), said retrieval module accessing said index file for comparing said text 
data request and said pattern word sets to identify a requested image identifier for 
locating a requested image from among said images stored in said memory device 
(figure 1, element 53). 

Regarding claim 44, Greene discloses a system for indexing electronic 
information, comprising: 

an authoring module that creates an index file that includes pattern word sets 
corresponding to data objects (column 5, lines 44-52); 

a speech recognition engine that converts a spoken data request into a text data 
request (converts speech to text; column 5, lines 2-5 with column 6, lines 1 -11); and 

a retrieval module that compares said text data request and said pattern word 
sets to locate a requested data object (column 5, lines 24-52). 

Regarding claim 45, Greene discloses a system for indexing electronic 
information, comprising: 

an authoring module that coordinates an authoring procedure for creating an 
index file that includes pattern word sets corresponding to data objects stored in a 
memory device (figure 1 , element 53), said pattern word sets being generated with a 
speech recognition engine that transforms spoken data descriptions into text data 
descriptions for creating said pattern word sets (converts speech to text; column 5, lines 
2-5 with column 6, lines 1 -11), said pattern word sets being associated with data object 
identifiers that identify said data objects (column 5, lines 24-52). 
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Regarding claim 46, Greene discloses a system for indexing electronic 
information, comprising: a retrieval module that manages a retrieval procedure in which 
a speech recognition engine converts a spoken data request into a text data request 
(converts speech to text; column 5, lines 2-5 with column 6, lines 1 -11), said retrieval 
module comparing said text data request and said pattern word sets to identify a 
requested object identifier for locating a requested data object from among data objects 
stored in a memory device (figure 1, element 53 with column 5, lines 24-52 and column 
6, lines 1-11). 

Regarding claim 47, Greene discloses an electronic indexing system 
implemented by: 

creating an index file that includes pattern word sets corresponding to data 
objects (column 6, lines 1-11), said pattern word sets being generated with a speech 
recognition engine that transforms spoken data descriptions into text data descriptions 
for creating said pattern word sets converts speech to text; column 5, lines 2-5 with 
column 6, lines 1 -11); 

utilizing said speech recognition engine to automatically convert a spoken data 
request into a text data request converts speech to text; column 5, lines 2-5 with column 
6, lines 1 -11); and 

comparing said text data request and said pattern word sets with a retrieval 
module to identify a requested object identifier for locating a requested data object 
(comparing word to clip; column 5, lines 24-42). 
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Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

7. Claims 9, 15, 29 and 35 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Greene in view of Gao et al. (PGPUB 2004/01 1 1272), hereinafter 
referenced as Gao. 

Regarding claims 9 and 29, Greene discloses a system for indexing electronic 
information, but does not specifically teach a system wherein said speech recognition 
engine performs speech recognition procedures upon said spoken data descriptions to 
produce said text data descriptions, said speech recognition engine being implemented 
as a large-vocabulary continuous speech recognizer that includes acoustic models, a 
dictionary, and a language model. 

Gao discloses a language translation and display wherein said speech 
recognition engine performs speech recognition procedures upon said spoken data 
descriptions (column 3, paragraph 0042) to produce said text data descriptions (text . 
words), said speech recognition engine being implemented as a large-vocabulary 
continuous speech recognizer that includes acoustic models (acoustic model; column 3, 
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paragraph 0032 with figure 1, element 110), a dictionary (figure 1, element 126), and a 
language model (figure 1 , element 112), to transcribe spoken words to text. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Greene's system wherein said speech 
recognition engine performs speech recognition procedures upon said spoken data 
descriptions to produce said text data descriptions, said speech recognition engine 
being implemented as a large-vocabulary continuous speech recognizer that includes 
acoustic models, a dictionary, and a language model, as taught by Gao, for audibly 
producing the natural language sentence in a target language (column 3, paragraphs 
0031-0032). 

Regarding claims 15 and 35, Greene discloses a system for indexing electronic 
information, but does not specifically teach a system wherein said speech recognition 
engine performs speech recognition procedures upon said spoken data request to 
produce said text data request, said speech recognition engine being implemented as a 
large-vocabulary continuous speech recognizer that includes acoustic models, a 
dictionary, and a language model. 

Gao discloses a language translation and display wherein said speech 
recognition engine (ASR) performs speech recognition procedures upon said spoken 
data request to produce said text data request (converting spoken words into text; 
column 3, paragraph 0032), said speech recognition engine being implemented as a 
large-vocabulary continuous speech recognizer that includes acoustic models (figure 1, 
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element 110), a dictionary (figure 1, element 126), and a language model (figure 1, 
element 112), to transcribe spoken words to text. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Greene's system wherein said speech 
recognition engine performs speech recognition procedures upon said spoken data 
request to produce said text data request, said speech recognition engine being 
implemented as a large-vocabulary continuous speech recognizer that includes acoustic 
models, a dictionary, and a language model, as taught by Gao, for audibly producing 
the natural language sentence in a target language (column 3, paragraphs 0031-0032). 

8. Claims 17-19 and 37-39 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Greene in view of Furuyama et al. (USPN 6,61 1 ,803), hereinafter 
referenced as Furuyama. 

Regarding claims 17 and 37 Greene discloses a system for indexing electronic 
information, but does not specifically teach a system wherein said retrieval module 
performs said search procedure by utilizing pre-defined priority indicators or pre-defined 
relationship indicators. 

Furuyama discloses a method for retrieving video and audio using indexed 
generated by speech recognition wherein said retrieval module performs said search 
procedure by utilizing pre-defined priority indicators (beginning/ending time) or pre- 
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defined relationship indicators (length; column 5, lines 4-60), to detect respective 
similarities. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Greene's system wherein said retrieval module 
performs said search procedure by utilizing pre-defined priority indicators or pre-defined 
relationship indicators, as taught by Furuyama, to detect respective similarities and 
generate video retrieval (column 5, lines 4-32). 

Regarding claims 18 and 38, Greene discloses a system for indexing electronic 
information, but does not specifically teach a system wherein said pre-defined priority 
indicators are used to evaluate search terms from said text data request based upon 
corresponding word lengths. 

Furuyama discloses a method for retrieving video and audio using indexed 
generated by speech recognition wherein said pre-defined priority indicators are used to 
evaluate search terms from said text data request based upon corresponding word 
lengths (length; column 5, lines 46-60), to detect respective similarities. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Greene's system wherein said pre-defined 
priority indicators are used to evaluate search terms from said text data request based 
upon corresponding word lengths, as taught by Furuyama, to detect respective 
similarities and generate video retrieval; column 5, lines 4-32). 

Regarding claims 19 and 39, Greene discloses a system for indexing electronic 
information, but does not specifically teach a system wherein said pre-defined 
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relationship indicators are utilized to evaluate search terms from said text data request 
depending one whether said search terms are located at the beginning or the end of 
said text data request. 

Furuyama discloses a method for retrieving video and audio using indexed 
generated by speech recognition wherein said pre-defined relationship indicators are 
utilized to evaluate search terms from said text data request depending one whether 
said search terms are located at the beginning or the end of said text data request 
(beginning/ending time; column 5, lines 4-32), to detect respective similarities. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Greene's system wherein said pre-defined 
relationship indicators are utilized to evaluate search terms from said text data request 
depending one whether said search terms are located at the beginning or the end of 
said text data request, as taught by Furuyama, to detect respective similarities and 
generate video retrieval; column 5, lines 4-32). 

Conclusion 

9. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

• Anderson (USPN 6,499,016) discloses automatically storing and presenting 
digital images using a speech-based command language. 

• Lux (PGPUB 2005/0137843) discloses a method and apparatus for authoring 
documents using object-based elements as an interaction interface. 
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• Gupta et al. (PGPUB 2005/0081 1 59) disclose a user interface for creating 
viewing and temporally positioning annotations for media content. 

1 0. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jakieda R. Jackson whose telephone number is 
571.272.7619. The examiner can normally be reached on Monday through Friday from 
7:30 a.m. to 5:00p.m. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, David Hudspeth can be reached on 571.272.7843. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 

JRJ 

May 8, 2006 




DAVID HUDSPETH 
SUPERVISORY PATENT EXAMINER 
TECHNOLOGY CENTER 2600 



